-
Notifications
You must be signed in to change notification settings - Fork 6.8k
fix bug in profiler tutorial when using cpu #13695
Conversation
try except approach only goes to ctx=mx.gpu() because test_utils.list_gpus() at least returns empty array and do not producing error
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
I think we are deprecating this method and instead use the GPU information from the context. I'm on my phone right now, so I can't give details, sorry |
@marcoabreu Is there any reference(link or alternative method name) of your thought? |
@marcoabreu I searched over list_gpus function and got latest one which was merged at Nov, 16.(#12918) and also using list_gpus util and they said fixed the util already. Is this fix related to your concern? |
I am not aware that we are deprecating this method otherwise LGTM. @marcoabreu can you provide details whenever you have time? Thanks! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
@marcoabreu - ping :-)
@marcoabreu any update? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@marcoabreu I think we can update the tutorial again when deprecation happens.
try except approach only goes to ctx=mx.gpu() because test_utils.list_gpus() at least returns empty array and do not producing error
try except approach only goes to ctx=mx.gpu() because test_utils.list_gpus() at least returns empty array and do not producing error
Description
When using cpu only, it produces error because test_utils.list_gpus() always returns at least empty array, and do not goes to the exception and context will be set as gpu. So I used if else statement to get context properly and it works fine.
For the reference below is the code of test_utils.py
python/mxnet/test_utils.py
and Error message
run_training_iteration(*next(itr))
Traceback (most recent call last):
File "", line 1, in
File "", line 5, in run_training_iteration
File "/home/ubuntu/anaconda3/envs/mxnet_p27/lib/python2.7/site-packages/mxnet/ndarray/ndarray.py", line 2135, in as_in_context
return self.copyto(context)
File "/home/ubuntu/anaconda3/envs/mxnet_p27/lib/python2.7/site-packages/mxnet/ndarray/ndarray.py", line 2084, in copyto
return _internal._copyto(self, out=hret)
File "", line 25, in _copyto
File "/home/ubuntu/anaconda3/envs/mxnet_p27/lib/python2.7/site-packages/mxnet/_ctypes/ndarray.py", line 92, in _imperative_invoke
ctypes.byref(out_stypes)))
File "/home/ubuntu/anaconda3/envs/mxnet_p27/lib/python2.7/site-packages/mxnet/base.py", line 252, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [03:59:09] src/ndarray/ndarray.cc:1270: GPU is not enabled
Stack trace returned 10 entries:
[bt] (0) /home/ubuntu/anaconda3/envs/mxnet_p27/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x21f5a4) [0x7fd2f7def5a4]
[bt] (1) /home/ubuntu/anaconda3/envs/mxnet_p27/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x21f981) [0x7fd2f7def981]
[bt] (2) /home/ubuntu/anaconda3/envs/mxnet_p27/lib/python2.7/site-packages/mxnet/libmxnet.so(mxnet::CopyFromTo(mxnet::NDArray const&, mxnet::NDArray const&, int, bool)+0x723) [0x7fd2fa8bc323]
[bt] (3) /home/ubuntu/anaconda3/envs/mxnet_p27/lib/python2.7/site-packages/mxnet/libmxnet.so(mxnet::imperative::PushFComputeEx(std::function<void (nnvm::NodeAttrs const&, mxnet::OpContext const&, std::vector<mxnet::NDArray, std::allocatormxnet::NDArray > const&, std::vector<mxnet::OpReqType, std::allocatormxnet::OpReqType > const&, std::vector<mxnet::NDArray, std::allocatormxnet::NDArray > const&)> const&, nnvm::Op const*, nnvm::NodeAttrs const&, mxnet::Context const&, std::vector<mxnet::engine::Var*, std::allocatormxnet::engine::Var* > const&, std::vector<mxnet::engine::Var*, std::allocatormxnet::engine::Var* > const&, std::vector<mxnet::Resource, std::allocatormxnet::Resource > const&, std::vector<mxnet::NDArray*, std::allocatormxnet::NDArray* > const&, std::vector<mxnet::NDArray*, std::allocatormxnet::NDArray* > const&, std::vector<mxnet::OpReqType, std::allocatormxnet::OpReqType > const&)::{lambda(mxnet::RunContext)#1}::operator()(mxnet::RunContext) const+0x110) [0x7fd2fa763ba0]
[bt] (4) /home/ubuntu/anaconda3/envs/mxnet_p27/lib/python2.7/site-packages/mxnet/libmxnet.so(mxnet::imperative::PushFComputeEx(std::function<void (nnvm::NodeAttrs const&, mxnet::OpContext const&, std::vector<mxnet::NDArray, std::allocatormxnet::NDArray > const&, std::vector<mxnet::OpReqType, std::allocatormxnet::OpReqType > const&, std::vector<mxnet::NDArray, std::allocatormxnet::NDArray > const&)> const&, nnvm::Op const*, nnvm::NodeAttrs const&, mxnet::Context const&, std::vector<mxnet::engine::Var*, std::allocatormxnet::engine::Var* > const&, std::vector<mxnet::engine::Var*, std::allocatormxnet::engine::Var* > const&, std::vector<mxnet::Resource, std::allocatormxnet::Resource > const&, std::vector<mxnet::NDArray*, std::allocatormxnet::NDArray* > const&, std::vector<mxnet::NDArray*, std::allocatormxnet::NDArray* > const&, std::vector<mxnet::OpReqType, std::allocatormxnet::OpReqType > const&)+0x3ca) [0x7fd2fa76ee5a]
[bt] (5) /home/ubuntu/anaconda3/envs/mxnet_p27/lib/python2.7/site-packages/mxnet/libmxnet.so(mxnet::Imperative::InvokeOp(mxnet::Context const&, nnvm::NodeAttrs const&, std::vector<mxnet::NDArray*, std::allocatormxnet::NDArray* > const&, std::vector<mxnet::NDArray*, std::allocatormxnet::NDArray* > const&, std::vector<mxnet::OpReqType, std::allocatormxnet::OpReqType > const&, mxnet::DispatchMode, mxnet::OpStatePtr)+0x839) [0x7fd2fa7748a9]
[bt] (6) /home/ubuntu/anaconda3/envs/mxnet_p27/lib/python2.7/site-packages/mxnet/libmxnet.so(mxnet::Imperative::Invoke(mxnet::Context const&, nnvm::NodeAttrs const&, std::vector<mxnet::NDArray*, std::allocatormxnet::NDArray* > const&, std::vector<mxnet::NDArray*, std::allocatormxnet::NDArray* > const&)+0x38c) [0x7fd2fa77512c]
[bt] (7) /home/ubuntu/anaconda3/envs/mxnet_p27/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2ab24f9) [0x7fd2fa6824f9]
[bt] (8) /home/ubuntu/anaconda3/envs/mxnet_p27/lib/python2.7/site-packages/mxnet/libmxnet.so(MXImperativeInvokeEx+0x6f) [0x7fd2fa682aef]
[bt] (9) /home/ubuntu/anaconda3/envs/mxnet_p27/lib/python2.7/lib-dynload/../../libffi.so.6(ffi_call_unix64+0x4c) [0x7fd3097dcec0]
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.